Skip to content

Report missed errors; pin hed-tests babc444 - #1501

Merged
VisLab merged 3 commits into
hed-standard:mainfrom
VisLab:address_issues
Nov 10, 2026
Merged

VisLab merged 3 commits into
hed-standard:mainfrom
VisLab:address_issues

Conversation

@VisLab

@VisLab VisLab commented Nov 10, 2026

Copy link
Copy Markdown
Member

Fix the false negatives in review items 8.1.1-8.1.7:

  • misnested parentheses are PARENTHESES_MISMATCH
  • a placeholder that is not the whole value is PLACEHOLDER_INVALID
  • curly-brace column names may contain any character
  • a definition beside other tags in a sidecar entry is DEFINITION_INVALID
  • a HED key nested in a sidecar column is SIDECAR_INVALID
  • units in a Def value are checked against the definition
  • sidecar strings that reference {HED} skip combination checks

df_to_hed no longer adds a Description tag beside definitions. Pin hed-tests babc444 and hed-examples de5c4b0.

Fix the false negatives in review items 8.1.1-8.1.7:
- misnested parentheses are PARENTHESES_MISMATCH
- a placeholder that is not the whole value is PLACEHOLDER_INVALID
- curly-brace column names may contain any character
- a definition beside other tags in a sidecar entry is
  DEFINITION_INVALID
- a HED key nested in a sidecar column is SIDECAR_INVALID
- units in a Def value are checked against the definition
- sidecar strings that reference {HED} skip combination checks

df_to_hed no longer adds a Description tag beside definitions.
Pin hed-tests babc444 and hed-examples de5c4b0.
@VisLab VisLab added bug Something isn't working submodules Pull requests that update Submodules code testing required-update labels Nov 10, 2026
@VisLab
VisLab requested a balanced review from Copilot November 10, 2026 19:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Reference handling can delete unrelated annotations or leave invalid HED for special-character column names, while overly broad Definition detection suppresses valid Description tags.

3 open issues
What changed in this PR

This PR addresses missed validation errors in hedtools and adjusts sidecar handling to distinguish standalone checks from checks that require assembled rows.

Changes:

  • Detect misnested parentheses, malformed placeholders, misplaced definitions, nested HED keys, and invalid Def-value units.
  • Broaden column-reference recognition and defer combination checks involving {HED} until row assembly.
  • Avoid adding Description tags beside definitions, with regression tests and changelog updates.
File Description
tests/​validator/​test_sidecar_validator.py Adds sidecar validation and reference regression cases.
tests/​validator/​test_hed_validator.py Tests parentheses nesting and placeholder placement.
tests/​validator/​test_def_validator.py Tests units supplied through Def values.
tests/​models/​test_definition_dict.py Updates the expected malformed-placeholder error.
tests/​data/​sidecar_tests/​both_types_events_with_defs.json Removes an ordinary tag beside a definition.
hed/​validator/​util/​string_util.py Detects misnested parentheses with equal counts.
hed/​validator/​sidecar_validator.py Checks definition placement and nested keys; defers {HED} combination checks.
hed/​validator/​hed_validator.py Reports placeholders that are not whole values.
hed/​validator/​def_validator.py Validates substituted values with definition-provided units.
hed/​tools/​analysis/​annotation_util.py Suppresses appended Description tags for definition entries.
hed/​models/​sidecar.py Uses the shared column-reference pattern.
hed/​models/​model_constants.py Defines a broader column-reference pattern.
hed/​errors/​error_messages.py Adds a positional message for misnested parentheses.
CHANGELOG.md Documents validation and annotation-conversion changes.

Review effort: Balanced

Comment thread hed/models/model_constants.py Resolved
Comment thread hed/models/sidecar.py Resolved
Comment thread hed/tools/analysis/annotation_util.py Resolved
- A brace name with a comma or parenthesis is not a column
  reference, since a reference must parse as one tag.
- Escape the reference in replace_ref's removal regex, so
  {my.col} and {col+1} remove only themselves.
- df_to_hed detects Definition tags, not any "definition/"
  substring, before leaving out the Description tag.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Definition detection misses namespaced definitions and misclassifies ordinary description values, while placeholder substitution corrupts hash-containing column references and produces invalid annotations.

1 open issue
1 previously missed issue

In code that hasn't changed since last review

Medium severity Preserve hash-containing references during value substitution

hed/​models/​model_constants.py:10

This now accepts # in a column name, but ColumnMapper._value_handler() replaces every # in the whole template before reference substitution. For the valid value template Label/#, {col#1} and cell 3, it produces Label/3, {col31}; subsequent substitution of {col#1} cannot find the reference, so assembly leaves invalid braces and loses the referenced annotation. Preserve curly-brace references while filling value placeholders, and add an assembly regression case for a hash-containing column name.

3 resolved since last review

Review effort: Balanced

Comment thread hed/tools/analysis/annotation_util.py Resolved
- Detect Definition tags with a library prefix and only at a
  tag boundary, so df_to_hed keeps Description tags on
  values that merely mention a definition.
- Leave "#" inside curly-brace references alone when filling
  a value placeholder, so {col#1} is still substituted.
@VisLab
VisLab merged commit 37a9551 into hed-standard:main Nov 10, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working required-update submodules Pull requests that update Submodules code testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants